09. Traffic Light Detection Node
Traffic Light Detection Node Overview
Once the vehicle is able to process waypoints, generate steering and throttle commands, and traverse the course, it will also need stop for obstacles. Traffic lights are the first obstacle that we'll focus on.
The traffic light detection node (
tl_detector.py
) subscribes to four topics:
-
/base_waypoints
provides the complete list of waypoints for the course. -
/current_pose
can be used to determine the vehicle's location. -
/image_color
which provides an image stream from the car's camera. These images are used to determine the color of upcoming traffic lights. -
/vehicle/traffic_lights
provides the (x, y, z) coordinates of all traffic lights.
The node should publish the index of the waypoint for nearest upcoming red light's stop line to a single topic:
-
/traffic_waypoint
For example, if
waypoints
is the complete list of waypoints, and an upcoming red light's stop line is nearest to
waypoints[12]
, then
12
should be published
/traffic_waypoint
. This index can later be used by the waypoint updater node to set the target velocity for
waypoints[12]
to 0 and smoothly decrease the vehicle velocity in the waypoints leading up to
waypoints[12]
.
The permanent
(x, y)
coordinates for each traffic light's stop line are provided by the
config
dictionary, which is imported from the
traffic_light_config
file:
config_string = rospy.get_param("/traffic_light_config")
self.config = yaml.load(config_string)
Your task for this portion of the project can be broken into two steps:
-
Use the vehicle's location and the
(x, y)
coordinates for traffic lights to find the nearest visible traffic light ahead of the vehicle. This takes place in theprocess_traffic_lights
method oftl_detector.py
. You will want to use theget_closest_waypoint
method to find the closest waypoints to the vehicle and lights. Using these waypoint indices, you can determine which light is ahead of the vehicle along the list of waypoints. -
Use the camera image data to classify the color of the traffic light. The core functionality of this step takes place in the
get_light_state
method oftl_detector.py
. There are a number of approaches you could take for this task. One of the simpler approaches is to train a deep learning classifier to classify the entire image as containing either a red light, yellow light, green light, or no light. One resource that's available to you is the traffic light's position in 3D space via thevehicle/traffic_lights
topic.
Note that the code to publish the results of
process_traffic_lights
is written for you already in the
image_cb
method.
Traffic Light Detection package files
Within the traffic light detection package, you will find the following:
-
tl_detector.py
This python file processes the incoming traffic light data and camera images. It uses the light classifier to get a color prediction, and publishes the location of any upcoming red lights.
-
tl_classifier.py
This file contains the
TLClassifier
class. You can use this class to implement traffic light classification. For example, theget_classification
method can take a camera image as input and return an ID corresponding to the color state of the traffic light in the image. Note that it is not required for you to use this class. It only exists to help you break down the classification problem into more manageable chunks. Also note that Carla currently has TensorFlow 1.3.0 installed. If you are using TensorFlow, please be sure to test your code with this version before submission. -
traffic_light_config
This config file contains information about the camera (such as focal length) and the 2D position of the traffic lights's stop line in world coordinates.
Helper Tool in the Simulator
In order to help you acquire an accurate ground truth data source for the traffic light classifier, the Udacity simulator publishes the current color state of all traffic lights in the simulator to the
/vehicle/traffic_lights
topic in addition to the light location. This state can be used to generate classified images or subbed into your solution to help you work on another single component of the node. The state component of the topic won't be available when running your solution in real life so don't rely on it in the final submission. However, you can still reference this topic in real life to get the 3D world position of the traffic light.
Using Different Classification Models for the Simulator and Site
We will test your team's code both in the simulator and on the testing site. Due to differences in the appearance of the site and simulator traffic lights, using the same traffic light classification model for both might not be appropriate. The
self.config
dictionary found in the
TLDetector
class of
tl_detector.py
contains an
is_site
boolean. You can use this boolean to load a different classification model depending on the context.